Evaluation criteria for district health management information systems: lessons from the Ministry of Health, Kenya

BACKGROUND The District Health Management Information Systems (DHMISs) were established by the Ministry of Health (MoH) in Kenya more than two decades ago. Since then, no comprehensive evaluation has been undertaken. This can partly be attributed to lack of defined criteria for evaluating them. OBJECTIVE To propose evaluation criteria for assessing the design, implementation and impact of DHMIS in the management of the District Health System (DHS) in Kenya. METHODS A descriptive cross-sectional study conducted in three DHSs in Kenya: Bungoma, Murang'a and Uasin Gishu districts. Data was collected through focus group discussions, key informant interviews, and documents' review. The respondents, purposely selected from the Ministry of Health headquarters and the three DHS districts, included designers, managers and end-users of the systems. RESULTS A set of evaluation criteria for DHMISs was identified for each of the three phases of implementation: pre-implementation evaluation criteria (categorised as policy and objectives, technical feasibility, financial viability, political viability and administrative operability) to be applied at the design stage; concurrent implementation evaluation criteria to be applied during implementation of the new system; and post-implementation evaluation criteria (classified as internal - quality of information; external - resources and managerial support; ultimate - systems impact) to be applied after implementation of the system for at least three years. CONCLUSIONS In designing a DHMIS model there is need to have built-in these three sets of evaluation criteria which should be used in a phased manner. Pre-implementation evaluation criteria should be used to evaluate the system's viability before more resources are committed to it; concurrent (operational) - implementation evaluation criteria should be used to monitor the process; and post-implementation evaluation criteria should be applied to assess the system's effectiveness.